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A METHOD AND SYSTEM FOR COMPLETE 3D OBJECT AND AREA 

DIGITIZING 

BACKGROUND OF THE INVENTION 

Several systems for generating computer models of objects, scenes, or terrain are known 
5 including those disclosed in United States Patent Nos. 6,094,269 (Den-Dove, et al.); 
5,513,276 (Theodoracatos); and, 5,497,188 (Kaye). These systems scan the surface of an 
object with a laser and digital camera combination to produce a wire-frame, or raster 
model for later testing or simulation purposes. Other systems may use a laser and video 
camera to gather 3D images for vircual reality applications. Thus, 3D measurements may 
10 be taken and digital images captured mih the objective of producing a non-life-like 3D 
model or a life-like 3D model consmicted with "real" digital images. In the non-Ufe-like 
3D case, the digital camera is used primarily to establish 3D coordinates in conjunction 
^ with a laser. In the life-like "real" image 3D model, the digital camera is used to boih 
establish 3D coordinates and to record actual images which will be incorporated in die 
15 resultant model- 
One shoncoming of the current systems is their inability to produce models of objects, 
scenes, and terrain that vary widely in size. Most known systems focus on either small- 
scale object modelling or very large-scale terrain mapping or scene modelling. These 
known systems are not flexible enough for applications demanding wide variations in the 
20 size of the subject matter for which a model is desired. This is especially so for systems 
ihat attempt to produce "real" image 3D models. 
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A need therefore exists for a method, system, and apparatus that will allow for the 
effective "real" image 3D modeling of subject matter that can vary widely in size. 
Consequently, it is an object of the present invention to obviate or mitigate at least some 
of tiie above mentioned disadvantages. 

S SUMMARY OF THE INVENTION 

The invention provides a s>^tem and apparatus for scanning areas and objects of varying 
size. 



In accordance with diis invention there is provided a method for acquiring visual 
information of an object, the method comprises a survey grid at a selected location, 
t 10 acquiring data points using a predetemiined survey plan of die site, calculating 3D 
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coordinates at the acquired data points, and subsequently post-processing this data for use 
by a variety of software applications, Accordingly, "real" 3D images of a target object or 
area can be collected and stored so that a user may subsequently regenerate and view 
images of the target object or area from any viewpoint via a computer and display. 

15 According to another aspect of the invention, there is provided an apparatus for 
goierating a three-dimensional data model of a surface comprising a data acquisition 
apparatos for scanning a surface to record digital images thereof and means for 
determining three-dimensional coordinates thereof 

The data acquisition apparatus comprises; at least one camera for recording digital 
20 images of the surface, the camera having an optical axis; at least two lascre for marking 
points in the digital images for determining the three dimensional coordinates of the 
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surface, the lasers having optical axes, the optical axes of camera and lasers being 
substantially parallel; a rail for mounting the camera and lasers, the camera being 
mounted between the lasers, and the rail having means for moving the camera and lasers 
along the rail; at least one post extending from die rack and atta^^ed to the rail by a 
means for rotating and horizontally shifting the rail, the post having means for shifting 
the tail; at least one moveable platform for mounting the posts and for positioning the 
camera and lasers proximate to the surface; and, data acquisition equipment for adjusting 
the platforms, the posts, the rail, the camera, and die lasers, for recordmg position data for 
the platforms, the posts, the rail, the camera, and the lasers, and, for recording the digital 



mi 



10 images. 



According to another aspect of the invention^ a data acquisition sysirai is provided for 
p generating a three-dimensional data model of a surface. This data processing system has 
'^1 stored therein data representing sequences of instructions which when executed cause the 
method described herein to be performed. The data processing system comprises: a data 
15 acquisition apparaws for scanning said surface to record digital images thereof and td 
record data for determining three-dimensional coordinates thereof; and, a data acquisition 
computer system in communication widi the data acquisition apparatus. The data 
acquisition computer system comprises: means for adjusting the data acquisition 
apparaws in accordance wifli user instractions; means for receiving position data and 
20 digital images from the data acquisition apparatus; means for determining three- 
dimensional coordinates of the surface from the position data and die digital images; 
means for associating the digital images with the diree-dimensional coordinates to 
produce a three-dimensional data model; memory for storing position data, digital 
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images, three-dimensional coordinates, and the three-dimensional data model; a display 
for presenting the three-dimensional data modd to the user, and, an inpm device for 
accepting user instructions from the user for adjusting ihe data acquisition apparatus. 

According to another aspect of the invention, a method is provided for generating a three- 
dimensional data model of a sur&ce. The method comprises the steps of: selecting data 
parameters for the three-dimensional data model; configuring a data acquisition system 
corresponding to the data parameters, wherein the data acquisition system comprises a 
data acquisition apparatus and a data acquisition computer system; scanning the surface 
with the data acquisition system to obtain digital images of the surfece and position data 
p 10 for detennining three-dimensional coordinates of the sur&ce; deteramung the three- 

fti dimensional coordinates of the surfece from the position data and the digital images: 

ii 

associating the digital images with the three-dimensional coordinates to produce ihe 
three-dimensional data model; and, storing ihe three-dimensional data model in the data 
acquisition system, 

15 BRIEF DESCRIPTION OF THE DRAWINGS 

The invention may best be understood by referring to the following description and 
accon^mying drawings which illustrate the invention. In the drawings: 

FIG, 1 is a simplified perspective view illustrating a single laser-camera scanning system 
in accordance with one embodiment; 

20 FIG, 2 is a perspective view illustrating the location of a single laser beam scan on a 
portion of a scene in accordance with one embodimojt: 
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FIQ, 3 is a detail perspective view illustrating the laser beam path from the viewpoint of 
the camera of the laser bean scan of FIG. 2 in accordance with one embodiment; 

FIG. 4 is a perspective view of a survey grid in accordance with the preferred 
embodiment; 

5 FIG. 5 is a decision tree diagram for planning a survey in accordance with the preferred 
embodiment; 

FIG. 6(a) is a schematic diagram illustrating the direction of an interior scan in 
9 accordance with the preferred embodiment; 

p FIG. 6(b) is a schematic diagram illustrating the direction of an exterior scan in 
P 10 accordance with the preferred embodiment; 
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FIG, 7 is a front view illustrating a data acquisition apparatus for large-scale surveys in 
accordance widi the preferred embodiment; 

FIG. 8 is a front view illustrating a data acquisition apparatus for small and medium-scale 
surveys in accordance with the preferred embodiment; 

15 FIG. 0(a) is a block diagram of a data acquisition computer system for implementing the 
method and controlling the data acquisition apparatus of the present invention in 
accordance with one embodiment; 

FIG. 9(b) is a schematic diagram of data acquisition computers configured as a web 
network in accordance with one embodiment; 
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FIG. 10 is a perspective view of a camera's field of vision cone in accordaace with ilie 
preferred embodiment; 

FIG. 11 is a diagram illustratmg calibration plate quadrants in accordance widi the 
preferred embodimem; 

5 FIG, 12 is a schematic diagram illustrating a horizontal (non-rotated) laser/camera group 
in accordance with the preferred embodiment; 

FIG, 13 is a schematic diagram illiistrating a non-horizontal (rotated) laser/camera group 
in accordance widi the preferred embodiment; 



pi FIO. 14 is a diagram illustrating a calibration plate wiA movemoit of a laser across the 
P 10 photograph for equal changes in distance to the target in accordance with the preferred 
embodiment; 
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FIG. 15 is a diagram illustrating laser scanning ranges in accordance with the preferred 
embodiment; 

FIG^ 16 is a table listing camera viewpoint angle degrees, distance cemre-line to 
IS viewpoint, and distance normalized for an example case in accordance widi the preferred 
embodimem; 

FIG. 17(a) is a table listing X, distance b, ratio a/{a^b\ and number of pixels from die 
centre line (CL), for an example case, in accordance with the preferred embodiment; 
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FIG, 17(b) is a graph illustrating the relationship between the distance to a target and the 
distance :from the centre of a photogr^h for an eicample case in accordance with the 
preferred embodiment; and, 

FIG. 18 is a flow chart illustrating a general m^od for collecting survey data including 
digital images and assoda^ 3D coordinates for use in generating "real" image 3D 
computer models of subject maner of varying size in accordance mth a preferred 
embodiment. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

In the following description, numerons specific details are set forth to provide a thorough 
understanding of die invention. However, it is understood diat the invention may be 
practiced without diese specific details, hi odwr instances, well-known software, circuits, 
structures and techniques have not been described or shown in detail in order not to 
obscure the invention. The terms "data acquisition computer system" and "post- 
processing computer system" are used herein to refer to any madiine fiw processing data, 
including the computer systems and network anrangements described herein. The term 
"real" image 3D modd is used herein to refer to a compute generated model that 
incorporates actual images c^tured by a digital camera, la the drawings, like numerals 
refer to like structures or processes. 

The invention described herein provides a method, ^tem, and apparatus for 3D 
scanning, acquiring and locating "real" visual information, storing this information with 
3D coordinates, and processing this information for use in a wide variety of applications. 
The invention provides a system and apparatus for scanning areas and objects of any aze. 
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The method of the present invention includes establishing a survey grid at a selected site, 
acquiring data using a predetennined survey plan and a data acquisition apparatus, 
calculating 3D coordinates, and subsequently post-processing this data so diat it can be 
utilized by a variety of software applications. Witii the mefliod, system, and apparaws of 
the present invention "real" 3D images from a target object or area can be collected and 
stored so ^at a user may subsequently regenerate and view images of the target object or 
area from any viewpoint scanned by the apparatus via a computer and display. 

In general, it is not possible to deiennine the 3D coordinates of objects within a 
photograph without an external ref^ence point. Typically^ a laser beam o^et from the 
position of the camera is used to provide such a reference point. Referring to FIG, 1, 
fberc is shown a simplified perspecnve view 100 illustrating a single laser/camera data 
acquisition apparatus HO in accordance with one embodiment of the invention- The 
single laser/camera data acquisition apparams 110 includes a laso: 120 and a camera 130 
which are directed toward a scene 140 a ponion 150 of which is to be scanned and 
photographed to produce a "real" image 3D computer model. In FIQ.l, the scene 140 is a 
room in a house. Both the laser 120 and camera 130 are directed parallel to each other 
along the y-axis of a reference coordinate system 160. 

Referring to FIG. 2, there is shown a perspective view 200 illusttming the location of a 
single laser beam scan 210 on a portion 150 of a scene 140 in accordance with one 
embodiment of the invention. Referring to FIG. 3, there is shown a detail perspective 
view 300 illustrating the laser beam path 310 fixra the viewpoint of the camera 130 of the 
laser bean scan 210 of FIG. 2, in accordance widi one embodiment of die invention, FIG. 
3 is close-up view of the beam path 210 shown in FIG. 2. FIG. 3 illustrates the apparent 
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shift in the laser poini of impingement 320, 330 fitom the viewpoint of die camera 130, 
This {^patent shift depends on the distance to the impinged object or scoie 320, 330. By 
locating the position of the laser 120 on the scanned image or photograph ISO, it is 
possible to determine the 3D coordinates of die impinged object 320, 330. This process is 
5 described in United States Patent No. 5,753,93 1 (Botchers, et al.). 

Method. Refencing to FIG. 18, tha-e is shown a flow chart 1800 illustrating a general 
method for collecting survey data including digital images and associated 3D coordinate 
for use in generating "real" image 3D computer models of subjea mait» of varying size 
in accordance with a preferred embodiment of &e invention. At step 1801, the method 
10 starts. At step 1802, the goal of the survey is defined. At step 1803, a survey plan is 
produced. At step 1804, a data acquisition systan is configured to conduct the survey. At 
st^ 1805, die survey is conducted using the data acquisition system and data is collected 
and stored. At step 1806, post-proc«sing is conducted on the data by a post-processing 
%| computer system to prepare die data for use by external client applications. At step 1807, 
1^^ 1 5 pos^p^ocessed data is exported to and used by external client applications. At step 1808, 
the method ends. 

Below, a detailed description of the method, system, and apparatus of the present 
invention is provided under the following headings: 

♦ Defining the Survey Goal 
20 • Producing a Survey Plan 

• Configuring the Data Acquisition System 



9 



17:19 FROM-Fasken Martlnaau DuMoulin LLP 



+416-384-7813 



T-808 P.019/07! 



• Conducting the Survey 

• Post-Processing the Data 

• Applying the Data 

Defining the Survey Goal. Prior to beginning a survey, the goal or purpose of the survey 
is established. Setting the right goal can reduce survey time and cost because the survey 
can be carried out mxh optimal settings. For example^ if the goal of a large scale interior 
survey of a stadium is to allow a computer game to provide a realistic view of the 
stadium from die viewpoint of players on the field, then it may not be necessary to scan 
details of each and every seat» from positions above, behind and below each seat, in the 
entire stadium. On the other hand, if a crime scene is to be transferred to an invesiigator*s 
computer for further analysis, then the amount of scanning detail required would need to 
be quite high, especially if the clue being sought is not known prior to scanning. In 
general, it may be bencr to acquire too much data rattier than too little as the time and 
expense to revisit a site may be significant and restrictive. Moreover, the final scan 
product can be filtered to contain only the data required. 

Producing a Survey Plan, After die goal has been determined, flie survey can be 
planned. Referring to FIG, 5, there is shown a decision tree SOO for planning a survey in 
accordance with the preferred embodiment. The scope of the survey is determined by 
parameters including scanning resolution, detail, accuracy, time, budget, scale, and site 
access. TTiese parameters may be interrelated. For example, low budgets may make it 
undesirable to attain a high scanning resolution. Site access restrictions may prevent 
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detail from being achieved in resmcied areas. If the goal of the survey is to obtain 
reconnaissance information, this will afifecr the specifications selected. 

Mode , The first step in developing a survey plan is to select the scanning mode. The 
scanning mode can include an interior mode 510 and an exterior mode 520. Exterior 
5 scans may be refezted to as 'object' scans. Interior scans are outward looking and can 
include 'object' scans as a sub-set of an overall site scan. Exterior scans are inward 
looking. For example, scanning all of the surfaces in a room involves pointing the 
^ scanning apparatus away from the inside of the room. Cta the odier hand, scanning an 
g object involves directing the scan towards 4e cenn-c of the object from outside the objea. 



I 10 Referring to FIG. 6(a), ihere is shown a schematic diagram 600 illusffating the direction 
p of an iwerior scan. The direction 610 of Ae scan is away fiom centre 620. Referring to 
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FIG. 6(b), there is shown a schematic diagram 630 illustrating the direction of an exterior 
scan. The direction 640 of the scan is towards centre 650. 

A survey can be conducted at night or in the dark by asing various spectral technologies! 
1 5 For example, infrared light can be used to acquire images without white light. 

Sfiglg. A next step in planning the survey is to select the overall size or scale of the scan 
530, 540- Overall scale can be divided into three subsets: large 550, 560, medium 551, 
561, and small 552, 562. Large-scale scans 550, 560 can be conducted for any area or 
object that can be •^seen" by ihe system. That is, the laser beams (or spectral sources) 
20 must impinge on the surface being scanned and flie cam^a (spectral or regular visual) 
must be capable of capturing the beams. Medium scale scans 551, 561 ai^ for areas and 
objects that can be reached but some resolution may need to be sacrificed in select 
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portions of the scene. Small-scale scans 552, 562 include areas and objecis that have the 
highest degree of survey design flexibility given the various survey constraints. For 
example, a bullet found at a crime scene may need to be scanned at a super-high 
resolution to detect minute striations allowing its model to be ''fired*' using a computer, in 
super-slow-motiou, thus allowing precise model studies. 

Data Quality , The quality of measurements can depend on time and budget limitations. 
Typically, the optimal survey configuration fliat can achieve the goals set for the survey, 
in the shortest possible time and at the lowest cost, is the ideal configuration. Data quality 
can be selected ftom high 570, medium 571, and low 572 resolution, accuracy, and/or 
1^ 10 detail. Note that modem digital cameras are often compared in terms of '^megapixels'* 
rather than resolution. For example, a three-megapixel camera may be considered as 
5? better than a one-megapixel camera. 



Compression and image stabilization methods may be used. Data compression may be 



6l used to ''fine tune" the operation of die survey. Image stabilization methods that allow 
15 images to be acquired witihioot degradation of picture quality may be used with caurion as 
flie positioning calculations, desaibed below, must take into account any "shifts" in 
picmre positioning caused by stabilizers. 

Having planned &e survey, the next step is to configure a system to implement the 
survey. 

20 Configuring the Data Aequiskion System. In cader to conduct the planned survey, a data 
acquisition system must be configured and calibrated. It is preferable that the data 
acquisition system allows for adaptation lo different survey scales and modes as defined 



12 



FROM-Fasken Martineau DuMoulin LLP 



+416-364-7813 



T-808 P. 022/075 



by the survey plan. The data acquisition system includes a data acquisition computer 
system and at least one data acquisition apparatus. The data acquisition computer system 
controls the data acquisition apparatus and uses data acquisition equipment associated 
with the data acquisition apparatus to &lfil its instructions. 

Data Acquisition Apparams > According to one embodiment of the inventiosn, the data 
acquisition system includes a data acquisition apparams consisting of posts and rails for 
mounting cameras and lasers. This system can achieve high quality for small objects (e.g. 
baseball sized) and relatively high quality for large-scale surveys (e.g. a stadium interior). 
According to one embodiment of this data acquisition apparatus, components axe 
mounted on a rail that can be raised using extendable posts. This data acquisition 
apparams can be transported via tracks, carts, ground rails, or hand 

Referring to FIG, 7, there is shown a front view 700 illustrating a data acquisition 
apparams for large-scale surveys 710 in accordance with a preferred embodiment of the 
invention. For large-scale surveys, the data acquisition apparams 710 can be mounted on 
ffucks 720, 721, or oAer vehicles, so that greater heights and stability can be achieved. 
Various leveling and measurement systems, including oif-tbe-shelf GPS (Global 
Positioning System) and range-finding equipment, can be incorporated into the data 
acquisition apparatus 710 to assure accurate measurements. The rail 730 is coupled to 
each post 740, 741 by means for shifting and rotating 750, 751, for example, hydraulic or 
electric acmators. Each post 740, 741 includes means for telescopic adjustment 760, 761, 
for example, hydraulic or electric acmators. The data acquisition apparams 710 can be 
dismantled for transportation. For example, the rail 730 may be composed of sections 731 
to facilitate disassembly and transport, 
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At least three cameras 771, 772, 773 can be used with each camera pointing in one of the 
following diiecdons: forward (into paper) 771, reverse (out of paper) 772> and up (above 
trucks) 773, Laser clusters 780, 781 can be configured so that the beams fi-om at least one 
pair of clusters 780, 781, located on eiflier side of the cameras 770, 771, 772, are captured 
by each one of the three cameras. The laser clusters 780, 781 are mounted on the rail 730 
on either side of the cameras 770, 771, 772. Each laser chister 780, 781 includes at least 
one laser. Each laser and laser cluster 780, 781 includes means for shifting 790, 791 
along the rail 730, for sample, hydraulic or electric acmators. 

Each vehicle 720, 721 is equipped widi data aajuisition equipment 795. The data 
acquisition equipment 795 facilitates communication witii a data acquisition computer 
system 900 which is described below. One of the vehicles 720, 721 can house the data 
acquisition computer system 900. 

A laser/camera "group" is composed of at least one camera and r^vo corresponding laser 
clusters. Typically, a laser/camera group consist of three cameras 770, 771, 772 and 
hence six corresponding laser clusters. In this case, each camera 770 and its two 
corresponding laser dusters 780, 781 may be referred to as a laser/camem "subgroup"- 
The camera and lasers of each subgroup point in the same direction as described above; 
The means for shifting and rotating 750, 751 along the rail 730 is used to shift each 
laser/camera group along the rail 730. 

Note that the means for shifting and rotating 750, 751 is used for supporting, shifting and 
rotating the entire camera/laser group. The means for shifting along the rail 790, 791 
includes means for varying parameters for each camera/laser subgroup. That is, each 
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laser/camera subgroup can have different settings depending on the survey site. For 
example, if Ae data acqnisition apparatus is located near a first building wall in tbe 
positive y-direction 160 which is to be scanned by a &st laser/camera subgroup and in 
die negative y-direction a conesponding second building wall is located somewhat 
further away and which is to be scanned by a second laser/camera subgrot^, then a 
greater "a-spacing", as described bdow, for die second laser/camera subgroup can be 
accommodated. 

The data acquisition apparatus 710 including rail 730, posts 740, 741, cameras 77X, 772, 
773, laser clusters 780, 781, data acquisition equipment 795, means for shifting and 
rotating 750, 751 means for telescopic adjustment 760, and means for laser shifting 790, 
791 are responsive, generally dwough the data acquisition equqimcnt 795, to a data 
acquisition computer system 900 whi(* will be described in more detail below. The data 
acquisition computer system 900 stores data including the positions of the trucks 720, 
721 on an established grid, the location and rotational position, of the laser/camera group, 
the spacing of individual lasers within each cluster (i.e. resolution related), and die "a- 
^acing" of die group (i.e. die distance between each camera and its corresponding lasers 
the significance of which will be described below). The data acquisition computer system 
900 also controls die survey speed and die rate at whidi photogt^hs are recorded. 

Rfifaring to FIG. 8, fliere is shown a fiont view 800 illusorating a data acquisition 
apparatus for small and medium-scale surveys 810 in accordance widi a preferred 
embodiment of die mvention. For small and medium-scale surveys, die data acquisition 
apparams 810 can be mounted on a single can 820 or platform. As widi die laige-scalc 
appaiaras 710, a data acquisition computer system 900 controls dements of die data 
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acquisition apparatus 810 generally through the data acquisition equipment 79S. A 
*'strobe" lifting system 830 can also be used if better lighting conditions are required to 
disxmguish the laser beams from the surface being scanned. 

Both trucks 920, 921 and carts 820 can be added in series to allow greater scanning 
depths by peimiiiing the laser clusters 780, 781 to be located fimher away from the 
cameras 771, 772, 773, In this way, the data acquisition system may include several data 
acquisition apparatus. The combination data acquisitions apparatus can then be 
transponed along each survey grid line as will be discussed below. 

Referring to FIG. 4, there is shown a perspective view 400 of a survey grid 410 in 
accordance with the preferred embodimmt of the invention. Each grid cell 420 rqpresents 
the possible location of a data acquisition apparatus 820. The width of the grid cell 420 is 
the same, or smaller to allow for overlap, as the laser/camera group's range of motion 
along the rail 730. The group moves along the rail 730 to its limit, like a typewriter 
carriage. The group is then ^'carriage-returned" as die whole cart 820 is moved to thi 
next grid cell 420. Higher resolution can be achieved with smaller camera movements 
and tighter laser clusters. The reverse can be done for lower resolution. The grid vertices 
430 can be established at the survey site using physical markings (e.g. using spray paint 
outdoors, chalk indoors, etc.) or GPS (Global Positioning System) data. 

An advantage of the vertical telescopic adjustment means 760, 761 is that it prevents the 
survey area 410 from becoming obscured- The rail 730, which includes means 750, 751 
for laser/camera group rotation, is mounted on top of the posts 740, 741. Guide wires, or 
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similar means, can be used to stabilize the data acquisition apparatus 710, 810, as 
required. 

Data Acquisition Computer System. Referring to FIG. 9(a), tbere is shown a block 
diagram of a data acquisition computer system 900 for implementing the method and 
5 controlling the data acquisition apparams of the present invention in accordance with one 
CTibodiment* The data acquisition computer systan 900 inchades a computer master node 
910 controlling a cluster of parallel computer slave nodes 920. These computers 910, 920 
may be mounted in a rack. They may be networked in a number of ways, including, but 
O not limited to cubes, hypCT-cubes (ie. cubes within cubes), meshes, and layered webs, 
*g 10 The data acquisition computer system 900 may also include an input device, memoiy, 
and a display. The input device may be a keyboard, mouse, tradd^all, or similar device. 

it 

^ The memory may include RAM, ROM, databases, or disk devices. And, the display may 

jl include a comimter screen or renninal device. The data acquisition compwer system 900 
%i has stored therein data representing sequences of instructions which when executed 
1^ 15 control the data acquisition f^ipaFatus and cause the method desciibed herein to be 
performed. Of course, the data acquisition computer system 900 may contain additional 
software and hardware a description of which is not necessary for und^tanding the 
invention. 

Referring to FIG. 9(b), there is shown a schonatic diagram of data acquisition computers 
20 910, 920 configured as a web network 950 in accordance witii one embodiment of the 
invention. Each layer of the web network can interact with another computer node. The 
connection between computers can be ait»ed depending upon the survey plan. For 
example, if required, one computo- fiom each layer can be connected direcdy to the 
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master node 910. That is, nodes la and lb may be conneaed directly 930, 940 to the 
master node 910. The 1-series of nodes controls each layer (ie. nodes 1, 1(a), and l(b)X 
This provides better control of the individual layers from the master node. Additional 
layers can be added as required. In addition, the number of nodes per layer may be 
varied* In FIG, 9(b), there are 8 nodes per layer (e,g- nodes 1, 2, 3, 4, 5, 6, 7, and 8 are in 
the first layer while nodes la, 2a, 3a, 4a, 5a, 6a, 7a, and 8a arc in the second layer). 
Moreover, some nodes (960 in FIG. 9(a)) can function as backups to take over in the 
event of the Mure of adjacent nodes. 

An advantage of having several nodes connected to each other is to allow for the '•smart" 



fti 10 processing of data. For example, if a node that processes a parricular laser (e.g, node 5b) 
^ acquires data that can not be used by another layer ^t processes its data (e.g. too dark, 
too much interference, etc), then the system should decide whether it is worth stopping 

5 s 

ji| the process or continuing. The only way a decision like tins can be made "on-the-fly" is 
to allow the interchange of «Tor-checking variables between the nodes at all times. If 
1 5 enough relevant eirois accumulate, then tiie software can calculate a conrctive couree of 
actiM, in real time, witiioui manual imervenrion. For example, data acquisition may 
continue if die loss of one laser is not deemed to adversely effect ibs resolution 
requirements of the survey plan and if time Umitarions diaaie that it is more important 
for the survey to proceed than to tenninaie. 

20 Another advantage of a parallel-processing environment is that extremely large amounts 
of data can be acquired and stored at extremely high rates. Each computer node canies 
out specific tasks in parallel with each other computer node. Consequently, the bottleneck 
in the scanning process will be die medianical systems used. For example, ilie speed of 
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the cameras (i.e. number of photographs per second) and the speed by which the 
laser/camera group can be moved or rotated along the rail, posts, and grid. 

In Oie web neiwoik configumtion of FIG. 9(b), tiie nodes in the outermost layer of the 
cluster (i.e. nodes lb, 2b, 3b, 4b, 5b, 6b, 7b, and 8b) control the data acquisition 
5 ^patams 710, 810 (i-e. cameras 770, 771. 772, lighting 830, laser clnsters 780, 781, 
group shifting and lotatiog means 750, 751, telescopic adjustment means 760, laser 
sliding means 790, 791, etc.) These nodes can be located at the data acquisition apparatus 
710, 810 in the data acquisition equipment 795. This "b-seiies" of nodes iiksiructs 

■ Ms 

Q individual data acquisition apparatus subsystems to cany out specific tasks. Theb-series 

hS 10 also acts as the gateway for incoming data. This incoming data is passed from the b-series 

111 

nodes to the "a-smes" nodes of ttie next layer (i,e. nodes la, 2a, 3a, 4a, 5a, 6a, 7a, and 
J 8a). The a-series cluster layer stores the data from the adjacent higher level b-series node 

^1 layer with time stamps and grid coordinates of die difTerent data acquisition apparatus 
%| subsystems (e.g. distance between laser chistere, distance between individual lasers in 
IS clusters, specnral values used fi>r each cluster, group rotation/position, etc). The "single- 
di^t" node layer (i.e. nodes 1, 2, 3, 4, 5, 6, 7, and 8) perfonns diagnostic services, 
synchronizes the clocks of all nodes, passes instructions regarding distances, an^es, light 
intensity, etc, from Ae master node 910 to die higher node layers. The roaster node 910 
provides user mput to all systems (e.g, via keyboard and display) and passes instructions 
20 to the various nodes to control the overall data acquisition system. 

The data acquisition computa sj^tem 900 can make use of a wide variety of parallel 
processing application development tools including "FADE" (pai^dlel applications 
development envuonmeni) and "XPVM" as well as code profiling tools such as Tau". A 

19 
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number of parallel processing libraries ai« becoming more common hence making 
parallel programming easier (e,g. "PAWS'' (parallel application workspace) and 
"POOMA" (parallel object-oriented methods and applications)). With the wide spread 
use of the bitemet and access to Linux programming expertise, these and other tools are 
being used more often. 

3D Data Coordinates: Dots and Stripes, As described above, the data acquisition 
apparatus 710, 810 includes laser clusters 780, 781 and cameras 770, 771, 772 mounted 
on a rail 730. For data acquisition, the camera '^pinhole" model can be used as described 
in United States Patent No. 4,979,815 to Tsikos, The hei^t of the laser/camera groups is 
conirolled with the telescopic adjustment means 760, 761. The distance between the 
individual lasers and between the laser clusters and cameras on die rail is oonttoUed by 
the laser shifting means 790, 791, In the present invention, the laser **dot" is always 
located along Ae centre horizon of the photograph or scene/object to be scanned. That is, 
the laser is projected paralld to the optical axis of the camem. 

A laser "snipe", pointing parallel to the camera direction, can also be used, where discrete 
segments of the stripe are equivalent to a series of laser dots with varying angles above 
and below tibe horizon. The laser stripe would also have unique solutions for various 
laser/camera a-spacings- Using a laser snipe may eliminate the need to rotate the 
laser/camera group. However, the use of laser stripes requires a relatively large number 
of calibrations and calculations to account for effects such as lens distortion and of&et 
viewpoints. Employing laser snipes may be worthwhile if the time required to rotate the 
group through its normal range of motion is greater than the time required ro obtain 
accurate laser stripe data. 

20 
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A description of a method for obtaining 3D coordinates of "real" images &om lasers and 
cameras in accordance with the preferred embodiment will now be provided. 

3D Data Coordinates: Deietmining the Location of the Camera Focal Point and the Field 
of Vision Angle . In the present invention, images are acquired by a digital camera. The 
camera can operate in the human-visual or spectral (ie. infra-red and other remote- 
sensing fi-equencjes not visible to naked eye) ranges. The camera has to be "calibrated" 
so diat the angles from the centre of the image to each pixd may be determined. 
Calibration tests can account for lens distortion and precision of the instrumentation. 

The camera "sees" an image within its field of vision, which is comparable to the view 
one wotild have looking throu^ a hollow cone from the narrow end. Rrferring to FIG. 
10, dicre is shown a perspective view 1000 of a camera's field of vision cone 10X0 in 
accordance with the prefeired embodiment. The total cone an^e may be referred to as ^ 
Note that an image is acmally inverted 1020 within the camera 1030 prior to being 
transferred to the negative film or recording surface. While the focal point 1040 may be 
located in the centre of the camera 1030, this is not always the case. Therefore, it is 
necessary to deteraaine where the focal point 1040 of the camera 1030 is located for the 
specific lens 1050 bang used. 

To calculate the true focal point 1040, at least two calibration tests should be made: one 
at a distance M+N, where N is approximately equal to JJf, and another at a distance N-. 
These distances may depend on the type of lens being used. A nawower angle lens would 
typically require larger distances (i.e. larger values of MmdN) whereas a wide-angle 
lens may use require relatively small distances. Photographs obtained using each such 
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lens would show two differem vantage points for the same calibration plate. The shift of a 
certain point 1060 at the outer edge of the photograph may be represented the vatiable O. 
The value of O can be obtained by reading the shift off the calibration plate 1070. The 
angle jll can be deiennined with the following equation: 

5 tan(fir2) = {W (equation 1) 

where O and M are both directly measurable ftom the calibration plate 1070. 

It is important to use Ae outermost point lOSO useable on the furthest photograph 1070, 
^ SO tiiat the true field of vision can be used. The value can be detenuined with the 

Isf?:?-- 

^ following equation: 

pi 

|l 10 tan(^/2)-/»/L (equation 2) 

isi where P and ^ /2 are both known, P being measured directly ftom calibration plate 1070. 

p From L the actual location of the focal point 1040 can tiien be determined. 

M 

H Note that ihe values of 0 and P can be convened to pixels per metre. This may be done 
by counting the number of pixels on the photograph included in the distances covered by 
15 O and These pixels per metre values will be required for las^ related calculations as 
will be discussed below. 

The calibration should be performed by the calibration plate 1070 at right angles to Ac 
direction of the camera 1030. Note that walls may be used as calibration plates. Also, die 
various calibration distances can be achieved by moving the group or data acquisition 
20 apparams relative to the calibration plate. Referring to FIG. 1 1, there is shown a diagram 
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illiistrating calibration plate quadrants 1100 in accordance with the prefeired 
embodiment. Perpendicularity can be verified by calculating die angles from all fi>ur 
quadrants 1110, 1120, 1130, 1140 around the calibration plate 1070. For 
perp«idicularity, &e angle 412 should be equal, widiin a given tolerance, for all four 
quadrants 1110, 1120, 1130. 1140. 

The camera viewpoint angles 1090 for each point widiin the camera fiame can also be 
calibrated. Therefore, every 'ting" 1170 widun the calibrated frame has a unique camera 
viewpoint angle 1090. Refening xo FIG. 16, there is shown a table 1600 listing camera 
viewpoint angle yr degrees 1610, distance centre-line to viewpoint 1620, and distance 
normalized {i.e. normalized such that J* = 1 unit at <l^2 = 29 degrees) 1630, for the 
example case wh^e 412 = 29 degrees and X - 1 m, m accordance with the preferred 
embodiment Knowing these viewpoint an^es 1090 is required for calculating the 
location of the laser reference point described below. 

3P Data Coordinates: Laser Beams . From die above, die viewpoint angle portions of an 
acquired image can be determined. To determine 3D coordinates of points widiin the 
image, an additional reference point is required. A laso- beam may provide diis actional 
reference point 

Each laser beam should be conical-shaped so that die furdier away it is from die data 
acquisiiion apparatus, die wider die beam is. Therefore, die laser should be of increased 
energy for objects located at greater distances. The angle of die cone should be chos«i to 
match die desired resolution. That is, feauires located further away may not necessarily 
require die same resolution as objects located nearby. If high resolution is required for 
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objects located further away, then a smaller cone should be used and/or the data 
acquisition apparatus should be moved closer to the taiget 

Referring to FIG. 12, there is shown a schematic diagram 1200 illustrating a horizontal 
laser/camera group 1210 in accordance with the preferred embodiment- In FIG, 12, the 
group 1210 consists of one camera 1030 and two lasers 1220, 1230. Th^ method of the 
present invention involves tiie use of a laser 1230 located a fixed distance "a" from the 
camera 1030. 

The laser 1230 points in a direction parallel lo that of the camera 1030, In other words, 
and referring back to FIG. 1, ^ = 0 degrees. Note that distance a can be varied 
depending upon ihe site and system characteristics which will be discussed below. 

Using the calibmtion plate 1070, calibrations can be made to ensure that ±e laser 1230 
and camera 1030 directions are in the same plane and the focal point 1040 of the camera 
1030 and hinge point 1240 of the laser 1230 are at the same y-coordinate (160 in FIG. 1)! 
In FIG. 12, the point of impingement 1250 of die laser beam (i.e. the laser dot) is located 
in the vertical centre of quadrant H (1121 in FIG. 11). If die second laser 1220 was 
calibrated, its point of impingement would be located in the vertical centre of quadrant IV 
(1141 in FIG, 11). It is important for the camera to be properly located and directed with 
respect to the calibration plates, as discussed above, and Hm the centre of die photo is 
identifiable by pixel coordinates. 

Noie that as the calibration plate 1070 is moved toward or away from the camera 1030 in 
the direction of die camera 1030 and laser 1230 (i,e. along the y-axis in die coordinate 
system 160 of FIG. 1), the laser 1230 points at exactly the same spot on die calibration 
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plate 1070, that is, a metres from the centre-line 1260, However, the camera 1030 *'sees" 
xtit laser point 1250 moving towards the edge 1270 of the photograph 12S0 as the 
calibration plate 1070 moves towards the laser/camera group 1210. If the direction of the 
laser is fixed at a different angle fiom the horizontal (i.e. if laser 1230 doesn't point in 
same direction as camera 1030), then the laser point of impingement 1250 would track at 
an angle from horizontal on the calibration plate 1070 as the distance between the plate 
1070 and the apparatus changes. In the case where the laser direction is paxalld to the 
camera direction, then the migration track of the laser beam from the vertical centre 1290 
can be used to check and calibrate the system as described below. It can be shown that for 
every identified laser point of impingement, the 3D coordinates of the coinciding portion 
of the image has a unique solution. In general, each calibration plate 1070 is flat. While a 
convex plate may seem more appropriate given the nature of the camera lens and 
changing angle of incidence, this is not necessary because these effects are accounted for 
by the calibrations described in association with FIGS. 1 1 and 16. 

3D Data Coordinates: Determination whm the Lasers are Pointed in Same Direction as 
the Camera a nd are Located at a Distance ''a'' from the Camera . Referring to FIGS. 10 
and 12, the goal at this point is to determine the viewpoint angle ^from the calibration 
plate 1070. To accomplish this goal, two pixel counts are required. The first being the 
20 number of pixels in the photograph 1280 between the centre line of the photo 1260 and 
the position of the laser a. The second count required is the distance P expressed in 
number of pixels. The ratio of a (in pixel counts) over F (in pixel counts) is a number 
that can be matched to the distance nomialized column 1630 of the table 1600 of FIG. 16. 

25 
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The nonnafeed column 1630 makes it possible to directly link laser positions with die 
image to camera viewpoint angles yr. 

The ratio a/P from the acqoired image is located in die distance noimalized colimm 1630 
of FIG, 16. The corresponding camera viewpoint angle 1610 from the table 1600 is the 
required viewpoint angle yr. Note again that table 1600 contained in FIG. 16 if for the 
example case where ^2 - 29 degrees. The table 1600 can be recalculated depending on 
die true range of vision for the selected camera and lens. 

Now that The camera viewpoini angle for the laser impingement point 1250 is kaown, 
die distance L along die y-axis (160 in FIG. 1) between the camera focal point 1040 and 



Uh 10 die impingement point 1250 can be calculated as follows: 

m 
m 

ill 

O dierefore, 

a/xmi^fi (equations) 

where « and are known and X can be expressed in metres. 

15 The computer system records this value of L along with the coiresponding x and z 
coordinates wluch correspond to the location of die camera focal point 1040 and die a- 
spacing in die X2 plane 160, In tiiis way, 3D coordinates are established. 

3D Data Coordinates: Determination when the Group Angle Varies Above or Below the 
Horizon in the YZ Plane. As mentioned above, it is simple and more accurate to rotate 
20 die entire laser/camera group 1216 than to use a rigid group with laser stripes. Referring 
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to FIG. 13, there is shown a schematic diagram 1300 illustraiing a rotated laser/camera 
group 1310 in accordance wiih flie prefeired onbodiment The group rotation an^e is 
given by Oy. Once the point of impingement 1250 is known with respect to die rotated 
group 1310, it can be projected to the main coordinate system 160. The x-cooidinate is 
imchanged because the group 1210 rotates about tiie x-axis 160. Howeva, the y and 2 
coordinates will change. To calculate the amount diat the y and z coordinates need to 
shift, deUaY and deltaZ, so that diey are referenced to the main coordinate system 160, 
the following calculations can be made: 



Sin(^)»(feAa2>Z 



jfl 10 therefor*, 

P 

dehaZ'LSinify) (equation 4) 



and 

Tan(^) = <feftaZ/C 
Aerefbre, 

15 C^(debaZ)/Tdai^) (equations) 

Which means, 

deltaT = C - £ (equation 6) 



By applying the shifts calculated in equation 4 and equation 6, it is possible to le-project 
the scanned image to the main coordinate system 160. 
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3D Data Coordinates: Determination for Movement of the Group Aloag the Y Axis fi.e. 
Movement of the Laser Across Photograph FrameV Referring lo FIG. 14, there is shown 
a diagram illustrating a calibration plate 1400 with movement 1410 of a laser beam 
aczoss the photograph 1280 for equal changes in distance to the target L in accordance 
5 with the preferred anbodiment In this case, the laser point of impingement 1250 moves 
across the photograph 1280 from the outside 1420 towards the centre 1430, For a parallel 
laser/camera group, the path 1410 is along the center horizontal axis 1290. Referring to 
FIG, 17(a), there is shown a table 1700 listing I 1710, distance h 1720, ratio alift-^b) 
1730, and number of pixels from the centre line (CL) 1740, for the example case where 

10 412 = 29 degrees. tan(^2) = 0,5543, a = 5 m, and the total pixel units from CL to edge « 
5000, in accordance with Ae preferred embodiment This table 1700 contains exemplary 
calculated values of the predicted location of the point of impingement 1250 on a 
photograph 1280 for equal increments of increasing distance between the camera 1030 
and the point of impingement 1250. Referring to FIGS, 12, 14, and 17, the coiresponding 

1 5 calculations are as follows; 

tan {412) = (fttbyi (equation 7) 

which is equivalent to 

* =Z tan i4l2) - a (equation 8) 

where I, ^2, and a are known from established settings and calculations described 
20 above- 
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The ratio «/(«+*) 1710 is then calculated. Note That when b becomes very large, this ratio 
^tptoacbes zero. The ratio afi»tb) 1710 can then be multiplied by die total number of 
pixels that span the photograph 1280 fiom the ceatre line 1440 to dte edge of the useable 
image 1420 to yield column 1740 in FIG. 17(a). Referring to FIG. 17(b), there is shown 
a graph 1750 illustrating the relationship 1760 between die distance to a tai^et 1710 and 
the distance fiom the centre of the photogr^h 1740 in accordance with the prefened 
embodiment. The "power series" curve 1770 is an exaggeration of the movement which 
makes it easier vo define the distance to target 1710 at which a new a spacing should be 
selected. Hence, it is more di£&cult to difTerentiate different valves of distance L when 
the distance a between the camera and laser becomes very small compared to the total 
distance photographed {a^b). For increa^g values of £ (i.e. increasing distance fiom 
the groiQj 1210 to the point of impingement 1250), Hie rate of change for the number of 
pixels crossed 1410 by the beam decreases. This means that a relatively large shift in L 
will begin to have a relatively small shift across 1410 the photogrsqph 1280 and therefore 
the calculated distance to the point of impingemoit 1250 will become more inaccurate. A 
solution to this problem is described below. 

3D Data Coordinates: Laser Cluster Configuration . In the present invention, the distance 
between a specific laser cluster 780, 781 and the camera's centie-line 1440 is detemiined 
by the distance X to the target, as will be described. As illustrated in FIGS. 12, 14, 15; 
17(a), and 17(b), the a-spadng between the camera 1030 and lasers 1120, 1230 should 
increase for greater distances Z to the target being scanned. Referring to FIG. 15, there is 
shown a diagram 1500 illustrating laser scanning ranges 1530, 1550, 1560 in accordance 
with the preferred embodiment, hi general, laser scanning should be limited to regions 
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1570 thai are greater than halfway from the ceoire 1440 of the photograph 1280. From 
FIGS. 17(a) and 17(b), this position if found by locating the value for the ratio 
«/(a+6)=0.5. The corresponding L valw, for teas ecample, is at 9 metres 1450 as shown 
in FIG. 14. 

5 The goal at this point is to have laser cluster beam points of contact 1510, 1520 located 
between the range limit lines 1570 for the entire movement of the group and data 
acquisition apparatus alcmg the grid in the x-direction (160 in FIG. 1 ). If a laser hnpinges 
on a suiSice outside the scanning range 1570, then it will be necessary to change the a- 

y ; "" ■. ; 

g spacing to a new range 1540. The laser moves across the photograph at the greatest rate 

Q 10 for changes in distance to target when it is nearer to ^e edge 1420 of the photograph as 

mi 

N' shown in FIGS. 14, 17(a), and 17(b). "Hierefore, greater accuracy in the calculation of 3D 

P coordinates is made possible. 



Each laser cluster 780, 781 may consist of many lasers. In this case, each laser can have a 
unique spectral signature or "colour" that can be recognized by the data acquisition 

15 computer system 900 or during post-processing of data. A method for recognizing the 
various spectral signatures of lasers is described in United States Patent No, 5,753,931 to 
Borchers, ct al. In the present invention, ihe spacing between lasers in a laser cluster 780, 
781 is related to the resolution required by the survey plan. That is, if 1-cm detail is 
required, then the lasers should be spaced no more than 1 cm apart. If 1-metre accuracy 

20 is sufficient, for large structures located fer from the apparams (e,g. buildings), then the 
lasers would be spaced less dmn 1 m apart on a large rail system 710 as shown in FIG. 7. 
While survey resolution is related to the spacing of individual lasers within a laser cluster 
780, 781, it is also rdaied to the size of the incremental '*srq)s" that each laser/camera 
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subgroup is shifted or rotated along the rail 730 for each photograph and to the size of the 
incremental steps of Ae entire data acqxusition apparatus 710, 810 as it moves along the 
survey grid 410. 



In FIG. 15, Range 1 1530 uses a small a-spacing 1540, whereas Range 3 1550, for further 
S t^ers, has a larger a-spadng 1540. The data acquisition computer system 900 can 
automatically adjust the a-spacing if the impingement area is beyond a predetermined 
range 1570, as described above. If high accuracy is required, more range-windows 1530, 
1550, 1560 can be allowed for at the cost of slowing the speed of ihe overall survey- 

To increase the speed at which a survey may be conducted, of course, repetitive 
p 1 0 mov^ents of the group along the rail, movement of the data acquisition apparatus along 
p the grid, eic,> can be automated- 

ii 

Conducting the Survey. Once the survey plan has been established and the data 
O acqinsition system has been configured, ftie survey may proceed and data may be 
0 acquired. According to the present invention, data acquisition includes the following 

IS elements; 

• Establishing the Survey Grid 



• Positioning die Data Acquisition Apparatus 



• Acquiring and Storing Data 



Establishi ng the Survey Grid . This is an imponant element in conducting the survey as an 
20 accurate, well laid out grid will maintain the overall quality of the survey data acquired. 
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It also allows the survey to be carried out in a structured and efficient manner^ A poorly 
laid out grid will degrade the integrity of the data, regardless of how well the r^ainder 
of the survey is performed. 

Referring to FIG. 4, the survey grid 410 need only be laid out on the floor or ground for 
the control of x and y directions 160. The z-direcdon 160 is controlled by the telescopic 
adjustment means 760 associated with each post 740, 741 of the data acquisition 
25>paratus 710, 810. Note also that fte **lines" definmg each cell 420 need not be drawn, 
rather, grid venices 430 may be marked with "dots" or pre-programmed using GPS 
navigation as mentioned above. 

A grid reference origin must be selected. For example, in FIG. 4, the origin 440 is 
located in the comer of the room. The reference system 160 may be used to define 
positive X, y, and z directions. 

The survey grid 410 should be established in all accessible locations in accordance with 
the resolution and detail of data required by the siirvey plan, A smaller data acquisition 
apparatus 710, 810 may be required to gather data from bdiind the table 450 shown in 
FIG. 4, for example, if information in this area is deemed important. Note that the survey 
grid 410 was not laid-out on top of the table 450 in FIG- 4. The reason being that pre- 
measured reference guides can be extended to known reference points where needed. 

Sub-grids can also be established in areas that require higher detail scanning. For 
example, if the data acquisition apparatus needs to be located on top of die table 450 so 
that greater detail can be obtained for the flower pot 460, then the data acquisition 
appararos can be referenced to the survey grid 410 laid out on the floor usmg devices 
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such as plumb-bobs and measuring tapes. Data acquisition system software can allow for 
the input of such parameters so that ihe acquired data can be referenced to the main 
survey grid 410, 

For small-scale surveys, the survey grid cell density may be as high as every few 
S centimettes. For medium scale surveys, the density could increase to half-metre intervals. 
For large-scale surveys, the grid could be established on parallel base lines and tie lines 
spaced, say, 40 metres apan, with reference points marked along the base and tie-lines at 
2-metre intervals. Fluorescent orange spray paint or chalk can be used to mark the grid in 
such circumstances. 



m 
PI 



10 Note that the data acquisition apparatus need not traverse the entire survey grid 410. 
Rather, the goal is to acquire data that can be used to display the various images acquired 
with 3D coordinates- The detailed grid layout gives the operator greater flexibility for 
deciding, on flie fly, where greater detail is required, and areas where the data acquisition 
apparatus need not traverse. For example, wide-open rooms need only be surveyed along 

IS an inside perimet^. 

Positioning the Data Acquisition Apparanis . The data acquisition apparams 710, 810 can 
begm scanning fipom any location on the survey grid 410 as long as die x, y and 2- 
coordinates are known and recorded for its every position. Otherwise, it will not be 
possible to reference Ae 3D position of the scanned surfiace to the main survey grid 410. 

20 The scanning process involves two classes of movement, namely, "primary movraient" 
and "secondary movement"- A primary movement involves the movement of the entire 
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m 



data acquisition apparatus 710, 810 over the main survey grid 410 or sub-grids. A 
secondary movement may include of the following four types of movement: 

1) Type 1; Changing 790, 791 the «i-spacing for each laser/camera subgroup; 

2) Type IL: Changing 790, 791 sq)aration between the individual lasers for each 
laser cluster; 

3) Type III: Verrical shifting 760, 761 of the laser/camera group and rail along 
the posts in the z-direction; and, 

4) Type IV: Horizontal shifting 750, 751 of the laser/camera subgroups along the 
rail. 



10 In general, primary movements are for s«up and data acquisition apparatus placement at 



ffi- 



the site. Secondary movemettts position the individual components of the data acquisition 
apparatus for detailed data collection. 

To be^ the survey, the data acquisition apparatus is setup and positioned at a grid 
reference point (e.g. (x,y,2) ^ (5 6 m, 0 m) in the suivey grid coordinate system 160). 
15 This is a primary movement Once the apparams is positioned, data can be acquired- 
Type I and Type II positions are set initially. Once these are set, either the Type III or 
Type rv movttnents can commence. Changes in Type I and Type II settings can be 
made as needed during scanning. The data acquisition computer system 900 keeps track 
of these settings and parameters. 
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Collecting and Sioring Data , As mentioned with reference to FIGS, 9(a) and 9(b), the 
data acqiusinon computer system 900 can conrrol the mechanical movements of the data 
acquisition apparams 710, 810 as well as the collection and storage of data. This is 
achieved with parallel processing software, which can be written in linisx. Each 
computer node of xhe data acquisition computer syst^ is synchronized and all data 
collected is stored with corresponding time stamps. Additional cross-re&rencing data can 
also be recorded for each parameter to allow for the more reliable post-processing of 
data. For example, semi-static settings associated with Type I and Type II movements 
can be recorded wi^ various data as required. 

As data is collected, it can initially be hdd in RAM /cache memory which saves time. 
From fliere, the data can be stored on a more pennanmt media (e,g, hard disk, CD ROM, 
etc.). Referring to FIG. 9(b), data is stored in the "a-series*' nodes (i.e. la, 2a, 3a, 4a, 5a, 
6a, 7a, and 8a) and can be backed-up to other '*a-sexies*' node or to a backup node 960. 
For example, a copy of data stored in node 8a can be made and transfexred to node 7a for 
backup, and vice- versa. Data backup is important as loss of data can involve substantial 
cost 

PoshFrocessing the Dm. After the survey is finished, data is transferred from the "a- 
s^es'' nodes of the data acquisition computer system 900 to an oflRce-bascd post- 
processing computer system. This post-processing computer system can be in the form of 
a network cluster designed as a series of layered nodes, controlled by a master node, 
similar to that illustrated in FIGS. 9(a) and 9(b) for the data acquisition computer system 
900, Again, each node can have a speciiic or redundant function. The goal of data post- 
processing is to compile all data for each sub-system (e.g., a-spadngs, main grid 
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coordinates, sub-grid coordmates, data acqmsition apparatus height, photographic 
images, laser frequendes^ etc.). Compiled data can then be stored in a database diat 
allows for rapid access and coirelation. For example, relational databases can be 
configured so that specific data parameters can be accessed by the post-processing 
computer system rapidly. In addition, different nodes can store different parameters all of 
which can be cross-referenced- 

During post-processing, each stored digitized photograph or image must be ''re-scanned'' 
by the computer. Re-scanning involves searching for ftie laser point of impingement and 
determining the 30 coordinates for each portion of the image. Methods of re-scanning are 
described in United States Patent Nos. 5,753,931 to Borchers, et al^; 5,513,276 to 
Theodoracatos; and, 5,675,407 to Ceng. 

In addition, 3D calculations are computed, as described above, and a maoix of solutions 
can be created that include the x,y,z position and pixel value for the hnage (e.g. 24 bit 
Red, Green, Blue sub-matiix). These manrices of solutions can be stored in various 
formats so that access by other applications is rapid. For example, the matrices can be 
brolcen down into low resolution blocks, medium resolution blocks, and hi^-resolution 
sectors. 

Moving-average pixel values can also be inooxpomted into the database- For example, 
images that are located at large distances firom the observer can be represented by the 
average value (e,g, RGB) of many pixels within a pixel-grouping radius. The greater the 
number of computer nodes that can be accessed in parallel by the post-processing 
computer system, the greater the ability to store the data in various formats. 
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The storage in the post-processing computer system of 3D positioning and image data in 
various databases fadKtates multiple applications each of which may require data of a 
cextain kinds or in certain formats. In addition, because it is likely that more data will be 
acquired than may be necessary for a given application, the post-processing computer 
system may include means for data screening and fihering. For example, data acquired 
&om the survey of a baseball stadiim^ can be divided into a number of 3D sectors each 
having varying levels of resolution and detail. These sectors can be analyzed, processed, 
and stored based on data requirements. A high detail sector can have more data stored 
whereas a low detail or low priority sector could contain more averaged data. 



Applying the Dam, The data acquired can be converted to the format required by a client 
application. Data compression techniques may be applied to the data to allow easier data 
transfer aaoss various media and storage devices. Applications of the data acquired by 
the meflxod, system, and apparatus of the present invention may include the following: 

• Small or large scale seme investigations independent of the time and place of 
data acquisition. 

• Roaming an area from a remote location (e.g. over a computer network). 

• Computer graphics and displays using "real'' images manipulated in 3D. 
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• Modification of moving picture scenes independent of the time and place of 
data acquisition. 

• Small and laxga object digitization. 

• Dangerous site exploration including outer space, nuclear reactors, and marine 
5 applications independent of the time and place of data acquisition. 

• Simulators, including fligjit simulau>rs, made more life-like with "real" 
images. 

• Full scans of movable objects, including people, for computer manipulation 
and animation using *'rear' images. 

10 t Model smdies and computer simulation testing based on the scanned data for 

real structures including airplanes, automobiles, ships, buildings, bridges, etc. 

To reiterate and expand, die method, system, and apparatus of the present invention 
includes the following unique features and advantages: 

• Flexibility for large-scale or small-scale surveys of objects or areas. 

15 • A systematic approach, from defining the goal of a survey, to making the 

resulting data available for a wide range of applications. 

• Acquisition of data efficiently and accurately without numerous calibrations 
and complicated calculations. 
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Identification of parameters that are important for configuring system (e,g. 
scale, resolmion, accuracy, mode, etc.)* 

Provision for transpoiting and mounting component assemblies (e.g. posts, 
rails, beams, etc) 

A computer architecture for acquiring, cross-referencmg, and processing 
extremely large data sets from multiple sources. 

A calibration method responsive to unique camera lens parameters (e,g. focal 
length, distortion, etc.). 

Responsive to accuracy reduction due to greater distances between target 
object and scanning apparatus (i.e, increased camera-laser separations) by 
defining "scanning ranges". 

Simple to implement, provides an entire practical process, and is not limited to 
**object'' scanning formanu&cturing and quality control. 

Takes advantage of parallel computing power to handle multiple lasers and 
camera groups. 

Use of a grid pattern for scanning. 

Capability of using range finders and similar devices to assist in defining 
camera-las» ranges. 
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• Accoirnnodation of many survey beam angles to the camera direction by 
movement and rotation of the ratire scamiing assembly. This greatly 
simplifies calibration reqmrements and increases flexibility. 

• Complete process from start to finish for any scale, resolution, and accuracy. 

• Parallel computer cluster for rapid and simple data acquisition and processing. 

• Las©c/camera "^groups" and mountingrtranspon systems (e,g. trades, beams, 
eta). 



Qi • Simple and accurate camera calibration procedure. 



• Det^ination of optimal camera-laser separations due to increased distance 
5* 10 to targets. 



Although ihe invention has been described with reference to certain specific 
embodiments, various modifications thereof will be apparttit to those skilled in the an 
without departing from the spirit and scope of die invention as outlined in the claims 
15 appended hereto. 
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